Rsyncrypto rsyncrypto https://rsyncrypto.lingnu.com/index.php?title=Main_Page MediaWiki 1.32.0 first-letter Media Special Talk User User talk Rsyncrypto Rsyncrypto talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk 0 1 1 2019-04-23T02:21:59Z MediaWiki default 0 wikitext text/x-wiki <strong>MediaWiki has been installed.</strong> Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software. == Getting started == * [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list] * [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ] * [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list] * [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language] * [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki] 5702e4d5fd9173246331a889294caf01a3ad3706 2 1 2019-04-23T02:31:42Z Shachar 1 Move page from old installation wikitext text/x-wiki =Rsyncrypto - Rsync Friendly File Encryption= =Latest News= == Latest Version - 1.13 == '''November 15^th 2016''' A mere 8 years after the previous one, a new version of rsyncrypto has been released. New to this release: * Refactored to work with OpenSSL 1.1.0 * Upgrade the source to be C++11 compilable, if not compliant. * --filelist determines whether to treat the name as a directory or a file based on whether the name ends with a slash. * --filelist now accepts as mandatory parameter the file list source. The first argument is now the directory in relation to which relative paths are interpreted. * Prevent absolute paths from appearing with two leading slashes when --trim=0 '''Windows support dropped'''. Due to problems translating the Visual Studio project files from version 2008 to version 2015, I have temporarily dropped Windows support for rsyncrypto. Offers of help would be greatly appreciated on the mailing list. -- Shachar == Site has moved == '''March 31^st 2012''' The site has moved to a new host. To reduce the upgrade costs, the content has been copied without the editing history. My apologies. -- Shachar == Spam Anti Measures Announcement == '''December 1^st 2008''' It is with heavy heart that I need to announce that from now on it will no longer be possible to edit the wiki unless you create an account and log in. It is still possible for everyone to do so, but the wiki will no longer allow anonymous edits. At this point it is not necessary to confirm your email address in order to create an account, but should the spammers become "smarter", we will do that as well. I'm sorry about this, but I don't have the time to baby-sit the wiki from automated vandalism. Anyone with a better solution is welcome to email the rsyncrypto mailing list. -- Shachar == Slashdot Comment Discussion == '''August 18^th 2008''' An article at slashdot raised the question of secure backup to remote server. Some of the comments suggested rsyncrypto, and some of the commenters raised some very good questions regarding the security of the algorithm and the implementation. The questions, as well as Shachar, the rsyncrypto developer, answers, may be considered an interesting read. * [http://tech.slashdot.org/tech/08/08/18/0033208.shtml Slashdot article] * [http://tech.slashdot.org/comments.pl?sid=648583&cid=24645975 Question about the effect of constant IV resets on patterns developing in the cipher text] * [http://slashdot.org/comments.pl?sid=648583&cid=24655977 Worries about the decision function used] == Latest Version - 1.12 == '''July 23^rd 2008''' New to this version: * Use safer method to create new files * On Windows - better error reporting You can get it from the SourceForge.net [https://sourceforge.net/project/platformdownload.php?group_id=129038 download page] =Why is Rsyncrypto= Sometimes it is necessary to store files on a remote server. This is typically needed for backup purposes. When that is done, there are two concerns that need to be addressed: # How to keep the privacy of the files stored? # How to keep bandwidth usage to a minimum? Both problems have rather simple solutions: # Encrypt the files prior to sending them. Keep the key locally. # Use [http://samba.anu.edu.au/rsync/ rsync] to only transfer the changes. There is just one problem - the two solutions contradict. Plain mode encryption of files hide the specific changes to the file, making rsync useless at detecting in-file changes. This is where rsyncrypto comes to the rescue. =What is Rsyncrypto= Rsyncrypto is a modified encryption scheme. It is based on industry standard AES for symmetric encryption, as well as RSA for having different keys for each file while allowing a single key to decrypt all files. It even uses an encryption mode that is based on CBC. Rsyncrypto does, however, do one thing differently. It changes the encryption schema from plain CBC to a slightly modified version. This modification ensures that two almost identical files, such as the same file before an after a change, when encrypted using rsyncrypto and the same key, will produce almost identical encrypted files. This means that both objectives can be achieved simultaneously. =Awards= Rsyncrypto has won first prize in the free software trophy competition ([http://www.tropheesdulibre.org/?lang=en Trophées du Libre]) in the security category. =Tutorials= There are some tutorials on line, which might help you with your experience with rsyncrypto: * [http://www.linux.com/feature/125322 Efficient rsyncrypto hides remote sync data] * http://blog.wuxinan.net/archives/86 = More Resources = * [[Todo|Plans for future versions]] * [[Algorithm|Details about the rsyncrypto algorithm]] f9e24977e5d2fbdf530ba627a6fd5b7e81ffbab0 0 2 3 2019-04-23T02:32:49Z Shachar 1 Move page from old installation wikitext text/x-wiki = Rsyncrypto Algorithm = Rsyncrypto was designed as part of the backup service offered by [http://www.lingnu.com Lingnu Open Source Consulting]. The challenges such a service poses require an answer to two basic needs: * Transfer incremental data over poor upload bandwidths * Encrypt the data in a way that does not allow the server to decrypt it Rsyncrypto solves this problem by performing an encryption that maintains a "bounded change" property - re encrypting the same file after a minor change in the plain text should produce a minor change in the cypher text. == Basics == In an attempt to reinvent as little as possible, rsyncrypto is based on industry standards wherever they apply to its unique requirements. In particular, broadly speaking, rsyncrypto uses the standard method of public key encryption, where each file gets its own, randomly generated, symmetric key (called "session key"), and that key is encrypted using a public key into the encrypted file. In addition to that, the algorithms used are as standard as possible. Rsyncrypto uses RSA for the public key encryption and AES for the symmetric key encryption. Further more, the basic schema of encryption is largely based on [http://en.wikipedia.org/wiki/Cipher_block_chaining CBC] mode encryption. == What isn't the same == Straight out CBC does not provide the bounded change property. When re encrypting a slightly changed plain file, the CBC-encrypted file will be different from the point of the change until the end of the file. This is where rsyncrypto deviates from the industry standard. In particular, rsyncrypto uses a decision function to look at a bounded window of history of plain text. Based on the history, the decision function will "trigger". It is important to understand that, because the decision function looks only at the plain text as input, even if the file changes in length, once the file before and after the change turn the same, sooner or later the decision function will start triggering at the same points along the data. This will happen even without having the old file, encrypted or otherwise, to compare against. This has the very desirable property that rsyncrypto needs very little state about each file in order to repeatedly encrypt it in an rsyncable way. Current version of rsyncrypto needs just 68 bytes of state (assuming 256 bit AES key) in order to encrypt the file again. The old version of the file itself need not be kept. Plain CBC uses the previous cypher block as a basis for encryption of the current block. This is the mechanism by which a change to the plain text propagates until the end of the file. Since the first encryption block does not have a "previous block", an initial vector (IV) is used as a pseudo previous block for the encryption. The entire rsyncrypto can be summarized with one sentence. Rsyncrypto uses the standard CBC encryption mode, except every time the decision function triggers, it uses the original IV for the file instead of the previous cypher block. = The Decision Function = In the current version, rsyncrypto uses the same decision function used by gzip. This triggers if the sum of the past 8196 bytes divides by 4096 with zero remainder. In other words, if <math>P_i</math> denotes the plain text at position i and n the current position, then the decision function will trigger if <math>\left( \sum_{i=n-8196}^n P_i \right) \equiv 0 \pmod {4096}</math> ca0f933dc41066ffe6fd663f2047891b227572c1 5 3 2019-04-23T02:49:29Z Shachar 1 make sure the equations are displayed wikitext text/x-wiki = Rsyncrypto Algorithm = Rsyncrypto was designed as part of the backup service offered by [http://www.lingnu.com Lingnu Open Source Consulting]. The challenges such a service poses require an answer to two basic needs: * Transfer incremental data over poor upload bandwidths * Encrypt the data in a way that does not allow the server to decrypt it Rsyncrypto solves this problem by performing an encryption that maintains a "bounded change" property - re encrypting the same file after a minor change in the plain text should produce a minor change in the cypher text. == Basics == In an attempt to reinvent as little as possible, rsyncrypto is based on industry standards wherever they apply to its unique requirements. In particular, broadly speaking, rsyncrypto uses the standard method of public key encryption, where each file gets its own, randomly generated, symmetric key (called "session key"), and that key is encrypted using a public key into the encrypted file. In addition to that, the algorithms used are as standard as possible. Rsyncrypto uses RSA for the public key encryption and AES for the symmetric key encryption. Further more, the basic schema of encryption is largely based on [http://en.wikipedia.org/wiki/Cipher_block_chaining CBC] mode encryption. == What isn't the same == Straight out CBC does not provide the bounded change property. When re encrypting a slightly changed plain file, the CBC-encrypted file will be different from the point of the change until the end of the file. This is where rsyncrypto deviates from the industry standard. In particular, rsyncrypto uses a decision function to look at a bounded window of history of plain text. Based on the history, the decision function will "trigger". It is important to understand that, because the decision function looks only at the plain text as input, even if the file changes in length, once the file before and after the change turn the same, sooner or later the decision function will start triggering at the same points along the data. This will happen even without having the old file, encrypted or otherwise, to compare against. This has the very desirable property that rsyncrypto needs very little state about each file in order to repeatedly encrypt it in an rsyncable way. Current version of rsyncrypto needs just 68 bytes of state (assuming 256 bit AES key) in order to encrypt the file again. The old version of the file itself need not be kept. Plain CBC uses the previous cypher block as a basis for encryption of the current block. This is the mechanism by which a change to the plain text propagates until the end of the file. Since the first encryption block does not have a "previous block", an initial vector (IV) is used as a pseudo previous block for the encryption. The entire rsyncrypto can be summarized with one sentence. Rsyncrypto uses the standard CBC encryption mode, except every time the decision function triggers, it uses the original IV for the file instead of the previous cypher block. = The Decision Function = In the current version, rsyncrypto uses the same decision function used by gzip. This triggers if the sum of the past 8196 bytes divides by 4096 with zero remainder. In other words, if <math>P_i</math> denotes the plain text at position i and n the current position, then the decision function will trigger if <math>\left( \sum_{i=n-8196}^n P_i \right) \equiv 0 \pmod {4096}</math> 70bf7a85dccbb3beaa883963f76f2131a0cdd07e 0 3 4 2019-04-23T02:33:28Z Shachar 1 Move page from old installation wikitext text/x-wiki =Future Plans= Rsyncrypto is currently running at version 1.xx. This version will not receive any more major upgrades. Version 2 is, currently, only in the planning, but the following features are planned for it ==Encryption== * Upgrade the decision function to something with cryptographic strength. Research is currently being done to try and figure out what should be done, and what performance and security implications it will have. * Change the RSA padding style so repeat encryption will produce identical headers == Performance == * Let he OpenSSL code encrypt more data in one go (currently encrypts one AES block at a time) * Find a way to utilize more than 1.5 CPUs concurrently == Modes of operation == * Support stdin&#x21d2;stdout encryption/decryption * Allow integration as a rsync "filter" * Allow password protected RSA keys * Eliminate the need for an external "gzip" command 6acfe5b338ec1cdf59a91e2dba6798dcb58663ae